新闻爬虫工程：把新闻采集做成可上线数据管道
SkillHub 社区 Skill「新闻爬虫工程」（@user_756344a1/news-crawler-engineering，v1.0.2）面向新闻与市场资讯爬虫的工程化交付。它规定先侦察列表 API 与分页再编码，选择 API/静态/逆向等采集模式，并以 get_market_news_info 与 get_market_news_content 双表建模。内容涵盖字段语义、正文抽取优先级、增量回填与按页入库、翻页停止逻辑、事务 upsert、三层监控报警、代理与内存注意点及验收清单。安装可通过 skillhub install 指向 Cursor 等 skills 目录，或按官方 install 文档让 AI 安装。
Read MoreCAU Portal (cau-portal): DSH Plugin and Data Pipeline for China Agricultural University Notices and Announcements
This is a DeepSeek Harness (DSH) plugin tailored for the China Agricultural University (CAU) context, providing an open-source crawler and AI processing pipeline. It supports sidebar panel display, parsing across multiple systems, AI-based summarization and classification, and deadline extraction, and offers MCP tools for direct querying within conversations. All data is fully self-hosted by users in private repositories, with no reliance on hosted services. Users need to configure a DeepSeek API key, a GitHub token, and a data repository, and can use the command line or GitHub Actions to periodically scrape and commit data, enabling efficient and autonomous management of notifications and announcements.
Read More